Peripheral Component Interconnect Express (PCI Express or PCIe) is a high performance, generic and scalable system interconnect for a wide variety of applications ranging from personal computers to embedded applications. PCIe implements a serial, full duplex, multi-lane, point-to-point interconnect, packet-based, and switch based technology. Current versions of PCIe buses allow for a transfer rate of 2.5 Giga bit per second (Gbps) or 5 Gbps, per lane, with up to 32 lanes. The PCIe bus protocol communication is encapsulated in packets. The packetizing and depacketizing data and status-message traffic is handled by the transaction layer of a PCIe port.
PCIe is used as a motherboard-level interconnect and an expansion board interface for add-in cards. For example, as illustrated in FIG. 1, a PCIe bus 100 interconnects the card 110 to the motherboard 120 and further connects expansion cards 130 and 140 through a host 160. The host 160 is connected to the motherboard 120. Thus, the PCIe bus 100 allows connectivity between the various cards to a CPU sub system 170 of the computing device. An expansion card is typically inserted into a slot. Usually, the host and/or the motherboard are referred to as PCIe roots and the cards are PCIe endpoints. An internal memory 180 is also coupled to the motherboard 120.
As illustrated in FIG. 2, the PCIe is a layered protocol bus, consisting of a transaction layer 210, a data link layer 220, and a physical layer 230. The PCIe implements split transactions, i.e., transactions with request and response separated by time, allowing the link to carry other traffic while the target device gathers data for the response. With this aim, the primary function of the transaction layer 210 is to assemble and disassemble transaction layer packets (TLPs). TLPs are used to carry transactions, where each TLP has a unique identifier that enables a response directed at the originator. The data link layer 220 acts as an intermediate between the transaction layer 210 and the physical layer 230 and provides a reliable mechanism for exchanging TLPs. The data link layer 220 implements error checking (known as “LCRC”) and retransmission mechanisms. LCRC and sequencing are applied on received TLPs and if an error is detected, a data link retry is activated. The physical layer 230 consists of an electrical sub-layer 234 and logical sub-layer 232. The logical sub-layer 232 is a transmitter and receiver pair implementing symbol mapping, serialization, and de-serialization of data. At the electrical sub-layer 234, each lane utilizes two unidirectional low-voltage differential signaling (LVDS) pairs for transmitting and receiving symbols from the logical sub-block 232.
Although the cards are physically connected to the motherboard, the PCIe bus protocol supports a hot-plug process, i.e., replacing system components without shutting down the power. This feature is highly important in blade servers where cards are frequently removed and inserted without powering on/off the server. A hot-plug process is supported in a current implementation of the PCIe buses and controllers. The PCIe standard defines a Slot Capabilities Register, Slot Control Register, and Slot Status Register to support the hot-plug process. These registers and the standard hot-plug process are described in detail in the PCI Express™ base Specification reversion 1.0a, sections 6.7, 7.8.9, 7.8.10, and 7.8.11 published on Apr. 15, 2003, by the PCI-SIG. The relevant sections are incorporated herein by reference in their entirety merely for the useful understanding of the background of the invention.
Generally, the Slot Capabilities Register identifies specific capabilities of a PCIe slot. With regard to a hot-plug, the Slot Capabilities Register includes several bits, two of which are: Hot-Plug Surprise bit that indicates that a card in a designated slot can be removed without any prior notification and Hot-Plug Capable bit that indicates that a designated slot is capable of supporting hot-plug operations. The Slot Control Register includes bits, that when set, define if a hot-plug interrupt can be asserted, e.g., if a power fault or a presence of card in the slot is detected. The Slot Status Register provides information about slot specific parameters, e.g., if a power fault or a presence of the card is detected. These registers will be referred to hereinafter as “Hot-Plug Registers” or “HPR”.
The hot-plug operation as currently implemented includes generating an interrupt when the slot status changes, i.e., from connected to disconnected and vice versa. The operating system (OS) captures the interrupt and allocates/reallocates resources to the inserted/removed device. Typically, when a card is inserted the OS enumerates the card in the order it appears on the PCIe bus.
Another type of interconnect bus that recently has been developed is a distributed interconnect bus, for example, a distributed PCIe bus. A distributed interconnect bus connects the root to endpoints over a distributed medium, e.g., a wireless medium, a computer network, and the like. The distributed interconnect bus includes two bridges that implement the PCIe protocol. A first bridge is coupled to the root and a second bridge is connected to an endpoint. The first and second bridge communicate over the distributed medium. An example of a distributed bus can be found in US Patent Application Publication No. 2009/0024782, entitled “Distributed Interconnect Bus Apparatus,” assigned to the common assignee, and incorporated herein by reference in its entirety merely for the useful understanding of the background of the invention.
Due to the physical nature of the distributed medium, the connectivity between the endpoints and the root is unreliable. For example, the wireless link may frequently be idle for a short period of time, and then operational again. Such an event may be treated as a hot-plug event (i.e., a card is removed and inserted). However, the above-referenced standard defines the hot-plug process in a standard PCIe bus where the root and endpoints are physically coupled to the bus, and the connectivity medium is entirely integrated in the computing device (e.g., server or PC). There is no a solution in the related art that provides a hot-plug process in computing systems that include distributed interconnect buses. Data transfers between the root and endpoint(s) are performed by encapsulating the TLPs in data structures compliant with the distributed medium. Further, the signaling definitions and protocol of a standard PCIe do not apply for communication over the distributed medium. Thus, if a bridge coupled to the endpoint generates a hot-plug interrupt signal, the signal cannot be transferred to the root (which informs the OS).
Therefore, it would be advantageous to provide a solution to support a hot-plug process in distributed interconnect buses.